A Gesture follower for performing arts

Our aim is to develop a gesture follower for performing arts, which indicates in real-time the time correspondences between an observed gesture sequence and a fixed reference gesture sequence (or in other words to time warp in real-time the observed sequence to the learned sequence). Applications concern for example choreography or (composed) music. The possibility to continuously establish correspondences between observed and reference gestures has important potentials. For example, this will enable the quantitative comparison of gesture data, at similar moments, between live and previously recorded performances. Particularly, this should facilitate the computation of parameters related to interpretation and expression. Moreover, this will also provide new tools to control and dynamically change mapping settings between gesture and digital media. The demo is organized as follows. After a presentation of related works, we describe the architecture of the follower and its current implementation. Third, we show results obtained with dance and an "augmented" violin. Finally we conclude by a discussion of our approach.